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OPTIMUM  SUBMARINE  ROUTING 

introduction  and  Summary 

The  purpose  of  this (paper  is  to  develop  numerical  rou- 

I. 

tines  to  determine  a  route  for  a  submarine  such  that  the  prob- 
ability of  its  being  detected  is  minimized.   The  basic  theory 
is  developed  and  a  numerical  routine  is  available  for  some  of 
the  simpler  problems. 

A  typical  problem  is  the  following.   The  submarine  is  at 
a  point  x0,yo?  which  we  may1  take  to  be  the  longitude  and  lati- 
tude,  and  it  wishes  to  go1  to  another  point  Xm,ym,  to  be  there 
at  a  specified  time  T„   The  probability  of  being  detected  in 
a  time  interval  of  length  At  is  f (x,y,u,v,w,t)At,  assuming 
that  it  has  not  been  detected  previously;   u  is  the  bearing 
angle  of  the  submarine,  v  is  its  speed,  w  is  its  depth, 
and  t  is  the  time;   f  is  a  function  describing  the  capa- 
bilities of'  the  hostile  observing  devices.   The  submarine  wishes 
to  take  a  route  such  that  the  probability  of  being  detected  en 
route  is  minimized. 

The  problem  is  an  extension  of  earlier  work  on  ship  routing. 
This  problem  is  more  difficult  than  those  considered  earlier  in 
several  ways.   First,  it  has  more  variables.   The  computational 
time  increases  roughly  as  the  square  of  the  order  of  the  system, 
and  the  numerical  routines  are  less  stable  as  the  number  of 
variables  increases.   Second,  the  problem  has  a  control  variable 
which  is  bounded.   The  depth  is  bounded  by  the  maximum  depth 


which  the  submarine  can  stand  or  can  go  at  any  position*  whether 
it  is  because  of  pressure  or  the  danger  of  running  aground.  It 
is  also  bounded  by  zero,  at  the  surface,  but  the  likelihood  of 
detection  usually  increases  there  and  automatically  eliminates 
any  difficulty.  Third,  most  of  the  data  is  empirical  and  must 
be  fitted  so  as  to  have  continuous  second  derivatives  with  re- 
spect to  the  variables  u,v,w.   Indeed  the  collection  and  fitting 
of  the  data  looks  like  the  biggest  single  remaining  difficulty. 
Fourth,  the  curves  may  have  corners;  that  Is,  there  may  be  points 
where  the  submarine  may  need  to  change  its  heading,  depth,  or 
speed  by  a  finite  amount. 

The  decision  was  made  to  use  the  method  of  variation  of  ex- 
tremals used  in  earlier  ship-routing  problems.   It  has  the  advan- 
tage of  flexibility  over  the  method  of  steepest  ascent  or  gra- 
dients, which  is  ill  adapted  to  problems  wherein  the  controls  are 
discontinuous  or  bounded.   It  was  originally  planned  to  make  the 
routine  very  general  so  that  it  would  handle  all  problems  that 
were  likely  to  occur.   It  Is  not  clear  now  that  this  is  feasible. 

The  problem  is  simplified  if  the  coordinate  set  is  chosen 
so  that  the  path  lies  near  some  great  circle,  say  the  great 
circle  connecting  the  initial  and  final  points.  Change  coordi- 
nates so  that  this  circle  becomes  the  equator.  Spherical 
,  coordinates,  such  as  longitude  and  latitude,  have  a  singularity 
at  the  poles,  where  longitude  is  undefined.   If  the  route  does 
not  srary  from  this  great  circle  by  more  than  about  400  miles, 
for  a  route  of  about  6,000  miles;,  maximum,  the  flat-earth 
assumption  Is  satisfactory  and  simplifies  the  computations. 
This  depends  on  the  fact  that  cos  Q     is  very  near  one  whenever 
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I9I  <  ,1  radian. 

The  theory  for  this  problem  is  not  readily  available,  if 
available  at  all,  and  it  was  necessary  to  include  a  discussion 
of  the  cases  where  the  control  lies  on  the  boundary  of  the 
region  of  allowed  values  and  where  corners  occur.   A  separate 
report  is  being  written  to  give  the  underlying  mathematics. 

Other  problems  of  a  new  type  were  encountered  and  studied. 
If  the  listening  device  is  accurately  located  and  if  the  water 
conditions  are  know  accurately  also,  it  may  be  possible  to  deter- 
mine blind  (deaf?)  areas,  regions  wherein  the  submarine  is  not 
likely  to  be  detected,  though  the  probability  of  detection  is 
large  at  neighboring  or  surrounding  areas.   The  boundaries  to 
these  regions  may  be  sharply  defined  and  the  corresponding 
probability  generating  function  f  is  then  discontinuous  in 
the  position  variables.   Some  time  was  spent  in  trying  to  develop 
the  theory  for  this,  with  limited  success;  with  this  exception, 
the  mathematical  theory  is  reasonably  complete. 


1.  Statement  of  a  Typical  Problem 

Let  us  consider  for  the  present  the  following  problem.  The 
submarine  is  to  go  from  one  given  point  xQ,y0  to  another  given 
point  x™,yT  by  a  given  time  T.   The  detecting  or  listening 
devices  have  capabilities  which  are  known  approximately;  we  have 
some  knowledge  of  their  distribution  and  we  have  made  measure- 
ments with  similar  devices  to  obtain  estimates  of  their  perfor- 
mance.  We  have  some  data  on  the  weather  and  have  estimated  the 
water  characteristics.   These,  together  with  studies  which  we  have 
made  on  similar  submarines  give  us  the  conditional  probability  of 
detection.   If  the  submarine  has  not  been  detected  previously,  the 
probability  of  detection  in  a  time  At  is  approximately 
f (x,y ,u,v,w,t)At ,  and  hence  the  probability  p  of  being  detected 
along  the  route  satisfies  the  equation  v 

dp  =  (l-p)f  dt. 
The  function  f  is  the  best  estimate  of  the  detection  capabili^ 
ties  of  the  hostile  observers,  as  made  up  by  engineers  and  intel- 
ligence, fitted  by  mathematicians  and  programmers.   For  the  present, 
no  distinction  is  made  here  between  detection  and  classlf lclation. 

The  above  equation  is  simplified  in  form  if  we  let 

(1.1)  z  =  -  ln(l-p); 
it  becomes 

(1.2)  z  =  f. 

We  are  Interested  in  long  routes  and  it  is  assumed  that  the 
time  required  to  ohange  depth  or  speed  Is  small  compared  to  the 
total  time  and  may  be  Ignored. 


2.  Equations  of  Motion 

Let  us  take  the  great  circle  containing  the  beginning  and  end 
points.  We  shall  treat  this  like  the  equator  ,  measuring  longitude 
x  In  radians  along  It  and  the  latitude  y  In  radians  normal  to 
It.   The  third  coordinate,  the  depth,  will  be  denoted  by  w.   The 
equations  governing  the  system  may  be  written  then 
v~cos  u 


x  = 


R  cos  y 
v  sin  u 


(2.1)     y  =    R 

z  =  f (x,y,u,v,w,t) , 
where  R  Is  the  radius  of  the  earth.  The  variables  x,y,z  are 
called  the  dependent  or  the  state  variables;   u,v,w  are  called 
control  variables. 

The  general  problem  Is  that  of  determining  the  control 
variables  as  functions  of  time  to  effect  the  desired  optimization. 
That  Is,  we  want  to  pick  the  heading,  the  speed,  and  the  depth  so 
that  the  submarine  will  get  from  the  specified  Initial  point   PQ 
to  the  specified  terminal  point  PT  with  z(T)  a  minimum.  The 
probability  of  being  detected  along  the  route  is,  by  (1.1) 

p(T)  =  1  -  expC-z(T)], 
and  we  want  to  choose  the  route  to  minimize  this. 


S^rYaUiational  Bquations,   Adjoint  Variables. 

Let  us  consider  any  route  and  a  neighboring  route.     We 

will  generate  the  neighboring  route  by  replacing     u,v,w    on 

the  original  route  by       u+lu,v+Sv,w+ Sw,   and  this  will  generate 

variations   (first-order  changes)    in     x,y,z     satisfying  the 

differential  equations 

v  cos  u   sin  y  t  v   sin  u  5  cos  u     ._, 

R  cos*y         **  '  R  cos  y  RcosyJV 

3.1)  i  Sy  =  £  cos  u  Su  +  ^  sin  u  Sv 

Sz   =  txSx  +  fyJy       +  fu5u  +  fySv  +  f^w  ; 

subscripts  indicate  partial  derivatives,  f  *  Bf/8x,  etc. 

Let  us  introduce  three  Lagrange  multipliers,   p,q,r  which 
are  unspecified  functions  of  t  so  far.  We  multiply  each  of 
equations  (3.1)  by  one  of  these,  add  and  integrate  from  0 
to  T.   We  may  write  the  result  as 

(3.2)    [T[p(Si  -  V  Tcosfy"  Y  «W  +  4#  +  r<^  -tjx   -fyJy)]dt 


-I 


cos  y 

The  terms  containing  the  variations  of  the  dependent  variables 
are  on  the  left;  those  containing  the  variations  of  the  control 
variables  are  on  the  right. 

Now,  on  the  left,  let  us  integrate  by  parts  those  terms  which 
involve  the  derivatives  of  the  dependent  variables,  so  as  to 
eliminate  their  derivatives  from  the  integral.   We  get  for 
the  left  side,  , 


(3.3)   Cp6x  +  qcy  +  r6z]T-  f T[6x(p  +  rf  )  +  6y(q  +  pY  COS  V**  y  + 

0  JO    .  R  cos  y 

+  iff  )  +  6zr]  dt. 
Now,  to  simplify  this,  let  us  choose  p,q,r  as  solutions  to  the 
differential  equations 


(3.4) 


fp  +  rfx  =  0 

4  +  pv  cos  u  sin  y  +  pf    0 
R  cos  y       y 


r  =  0. 

Equations  (3.4)  are  called  the  adjoint  of  the  variational ^equa- 
tions (3.1).  When  Eqs.  (3*3)  and  (3.4  are  used,  (3.2)  reduces  to 

(3.5)    [P6x  +  q&y  ♦  vbz]     =    fT[(-p  f-f|^  4  4  XJgUl  +  Pfu)6u  4 

+  <P  r§§ry  +  1  ^F1  +  rV6*  +  <*„«*]«. 

This  is  the  Important  formula  that  gives  us  a  relation  between 
the  end  values  of  6x,6y,6z  and  an  Integral  involving  the  varia- 
tions of  the  control  variables  6u,6v,6w.   It  is  important  that  we 
do  not  have  to  bother  with  the  interim  values  of  6x,§yf6z. 

It  is  often  convenient  to  consider  the  above  equations  as 

vectorial.   Let  X  =  xi  +  yj  +  zk,  P  =  pi  +  qj  +  rk,  and 

h  /;i     o"   v  cos  u  ,    v  sin  u  ,    -. 
(3*6)     F  =  R  cos  y  1  +  R J  +  fk* 

Eq.  (3«5)  then  becomes 

(3.7)  [P-6X]J  a  J  P.(Pu6u  +  Py6v  +  Pw6w)dt. 

Comments:   In  the  above  derivation,  we  have  assumed  that  there 
are  no  corners.  A  corner  is  a  point  t,   where  any  of  the  control 
variables  are  discontinuous.   If  there  is  a  corner  at  t,  ,  and  if 
we  allow  t^  to  vary,  then  a  term  must  be  added  on  the  right 

(3.8)  -Of]/  dt,; 

cl"  1  7 


there  Is  one  such  term  for  each  corner. 

It  is  also  assumed  that  f  has  continuous  derivatives 
for  the  moment.  This  is  not  the  case  in  some  cases  of  interest, 
but  iteseems  simpler  to  discuss  this  extension  afterward. 

Index  notation.   We  will  use  the  vector  notation  given 
earlier  whenever  vector  properties  are  to  be  stressed.   It  is 
also  convenient  to  use  index  notation,  as  follows.   Let  x,y,z 
be  denoted  by  xi,x2,x3,  or  x  ,  p,q,r  by  Pi,P2»Ps,  or  p,, 
Hff-*,  2-^S-a,  t    by  f*,4*«,f«,  or  f1,  and  u,V,w  by 
u1,u2,u8  or  u  •  The  use  of  superscripts,  rather  than  subscripts 
is  common,  particularly  in  differential  geometry.  In  general, 
the  state  variables  and  the  Lagrange  mutlipliers  will  have  Latin 
indices  and  the  control  variables  will  have  Greek. 

We  will  indicate  partial  derivatives  by  indices 

(3.9)    f*  =  efVex^,  f^=  fffVeu0"  , 

etc.  In  this  index  notation,  equations  (2.1),  (3.1),  (3.4), 
(3.5)  may  be  rewritten 

(3.10)  x1  =  f1 

(3.11)  Jx1  =  lfj*xJ  +  If^u0* 

(3.12)  p±  =  -  zf^P;j 

(3.13)  [ip^x1]*  =    J      1   p^u^dt 

It  is  understood  that  the  range  of  both  Latin  and  Greek  indices 

will  be  1,2,3,  and  if  no  confusion  seems  likely,  the  index  of 

summation  will  not  be  written.    This  notation  simplifies  the 

writing  of  equations,  but  no  notation  eliminates  the  actual  d etails 

of  substuting  the  actual  expressions  into  the  various  equations, 

carrying  out  the  differentiation,  etc. 
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Particular  solutions  to  the  adjoint.  We  may  pick  , partic- 
ular solutions  to  the  adjoint  so  that  we  get  the  effects  on 
various  state  variables  of  changing  the  control  variable; 
G.  A.  Bliss  did  essentially  this  in  Ballistics  at  Aberdeen 
Proving  Ground  during  World  War  I.   If  we  pick  a  particular 
solution  P1   to  the  adjoint  with  the  pro pertyn that t  "p^T)  m   i, 
then  equation  (3.5)  reduces  to 

T 
10 


(3.14)    ix(T)  =  (P*-S3&0  +  f  iP^F^Ju^dt; 


that  is,  this  particular  solution  gives  us  the  first-order 
effect  on  x(T)  of  changing  the  control  variables.   Similarly, 
if  the  components  of  P   are  such  that 

(3.15)  p*(T)  =  S*  =(  1      if  i  =  j 

■  0      if  i/j, 

then 

f  T 

(3.16)  Jx^T)  =  J  iP^F^u^dt  +  (Pi.^5f)0  . 

Note  that  the  subscript  on  the  p's  indicates  the  component 
and  the  superscript  indicates  which  state  variable  it  is  asso- 
elated  with. 

Finally,  if  we  have  a  function  g(x,y,z)T  ,  to  be  evaluated 
at  the  end point  of  the  route,  we  can  find  its  variation  as 
•follows.  Let  us  pick  a  particular  solution  P_  to  the  adjoint 

© 

such  that  at  time  T, 

(3.17)  (Pg)t=?  -  Vg, 

gradient  g  being  evaluated  at  the  endpoint  of  the  route. 
For ,  then , 


(3.18)    Sg   =  iS-Sr  £xJ 


bx% 


.5  Pa 


=  (Pg«*X)T  (by  choice  of  Pg$ 

■  (Pg-SX)0  +  f  iPg.iP^Su^dt      (by  (3,7))/ 

We  will  usually  have  problems  wherein  the  initial  point  Xq  is 
given  so  that   X^  =  0  and  the  first  term  in  the  last  ex- 
pression will  drop  out.  On  the  other  hand,  we  will  consider  some 
problems  where  the  terminal  time  T  is  not  specified.  In  this 
case  another  term  must  be  added;  we  will  take  this  up  when 
the  time  arises. 
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ki   Condition  for  an  Extremum 

In  this  section  the  first  necessary  conditions  for  an  ex- 
tremum will  be  given;  they  will  be  established  later. 

We  require  of  any  route,  to  be  admissible,  that  x(0)  «  0, 
y(0)  =  0,  x(T)  -  xT,  y(T)  =  y?  ,  and  we  take  z(0)  »  0.  The  dif- 
ferential equations 


(2.1) 


V  cos  u 
fx   ■  R  cos  y 
*   v  -sin  u 
y  "    R 


^z  =  f 
must  be  satisfied.   In  addition,  the  depth  w  must  satisfy  the 
condition 
(411)     0  <  w  <  ,WmaX 

where  W    is  the  maximum  depth  the  submarine  can  run.   In 
max 

addition,  the  speed  v  is  also  limited, 
(4.2)     0  £   v  <  Vmax  . 

with  V_„  a  constant  of  the  submarine,  or  perhaps,  a  function  of 

msx 

the  depth.  We  are  assuming  that  the  additional  noise  generated 
whenever  v  =  V"  „  rules  out  v  =  V*   -  but  there  may  be  problems 
where  the  optimum  solution  will  require  that  v  =  ^maxi  sfitthat  it 
should  go  at  maximum  speed,  the  reduced  time  more  than  offsetting 
the  increase  in  noise. 

A  route  which  satisfies  the  constraints  above  will  be  cal- 
led admissible.  A  set  of  control  variables  which  are  piecewlse 
continuous  and  which  satisfy  the  constraints  (4.1)  and  (4.2) 
are  called  allowable.  Allowability  is  a  local  constraint  or  con- 
dition on  the  control  variable. 
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We  seek,  among  all  admissible  routes,  the  one  such  that 

y 
z(T)  »  minimum.  Usually  there  is  Just  one  such  route. 

Minimum  Principle,  Euler  Equations.   It  is  a  very  general 
principle  that  the  optimum  route  must  satisfy  the  condition 
that  the  control  variables  must  be  chosen  so  that  they  maximize 

(4.3)  H  =  P.P 

compared  with  all  allowable  controls,  for  all  t,   0  <  t  <  T, 
for  some  solution  P  to  the  adjoint.   (This,  with  a  change 
of  sign,  is  the  Pontryagin  maximum  principle.)   H  is  called 
the  Hamlltonlan.   The  difficulty  Is  that  in  P  are  three 
constants  of  integration  associated  with  the  adjoint.   We  may 
write 

(4.4)  P  «  c^1  +  c2P2  +  c3?3  , 

the  P's  being  defined  earlier  (see  (3«l5))« 

We  can  choose  one  relation  among  the  c's;  the  most 

convenient  is  to  choose  c~  to  be  1   (one).  Then  the 

other  two  must  be  chosen  so  that  x(T),y(T)   assumed  the 

desired  values  x„,,yT. 

If  the  values  of  u,v,w  which  minimize  H  lie  interior 

to  the  domain  of  allowed  values,  then  the  Euler  equations  are 

a  necessary  condition 

(4.5)  flH/eu^s  0  =  P'dP/du^ 

or,  explicitly, 

_  v  sin  u        ■    v  cos  u  -         n 

1  R  cos  y  +  q  R +  rf u  =  ° 

V>-6)        \  P  B-?§§4  +  q  2^  +  rfv  =  0 
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COS 

u 

+  q 

p  a 

COS 

y 

fw 

= 

0. 

1       2     1  12     1 

where   p  »  o,p  +  o2p  +  p  ,   q  «  o1q  +  c2q  +  q-',  r 


1*   T  ' 


c,r  +  0or  +  r-  • 


The  Euler  equations  and  the  adjoint  system  together 
are  here  called  the  Euler-Lagrange  equations. 

It  may  be  that  H  Is  minimized,  not  by  an  Interior 
value  of  w  but  by  w  on  the  boundary  of  the  allowed  values , 

<*•>>     ».=  Wmax 

for  part  or  all  of  the  route.   This  is  probably  the  case 

whenever  the  search  effort  is  mostly  surfaoe  and  aerial. 

In  this  case,  the  last  of  equations  (^.6)  is  replaced  by  (4. 7), 
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5»  The  Numerical  Routine 

In  this  section  the  fcofc tine  for  determining  the  route 
Is  discussed. 

Let  us  guess  a  set  of  values  for  c, ,c2  .  (This  dholce 
may  pose  a  little  problem  but  has  not  in  any  problems  run 
so  far).   Let  us  run. the  route,  determining  the  control 
variables  for  each  value  of  t  by  the  minimum  principle. 

Let  us  now  consider  the  effect  on  the  control  variables 
of  changing  c,  ,c2  by  a  small  amount.   What  change  would 
this  effect  In  the  control  variables,  assumlngwe  are  at 
any  point  of  the  route?  The  Euler  equations  (^.5)  must 

hold  on  the  original  and  the  varied  path,  and  hence 

2  2 

(5.1)  2T-~r  dc.  +I-~—  6iT  =  0     1-1,2,  since  c~  =  1. 

M  ixri*^    *  rdu°eux  «* 

The  first  of  these,  C  =  1  ,  Is-,  when  expanded, 

t  <  o\            t   -lv  sin  u    1  v  cos  u  .  „1 ;"#  *  - 
(5'2)     ("p  B  cos  y  +  q  B +  r  V  d?l 

(    ^2  v  sin  u    2  v  cos  u    2  r    xAn 
+  ("p  B  cos  y  +  q  B +  r  fu)dc2 

(   *   v  cos  u   n   v  sin  u    .  r     %  . 
+  (~p  B  cos  y  "  *  B +  r  fuu5  6u 

(   «  sin  u    „  cos  u   „  -   *  ,_, 
*  ("p  B  cos  y  +  *  ~T~  +  r  fuv}  6v 

'   +  r  fuw  6w  =  0; 

the  second  Is,  c  =  2, 

tc  i\  i   «1  cos  u     1  sin  u    1-  |-^i 

(5.3)  (  p  B  cos  y  +  q  ""B-  +  r  fv)dcl 

/  JZ     cos  u    „2  sin  u   „2  ^  *._ 
+  *  p  B  cos  y  +  q  ~B~  *  r  fv)d02 

i   r*     sin  u   .    cos  u   _  -   >A„ 
+  ("p  B  cos  y  ♦  q  ~B~  +  r  fvu)6u 

+  rf^y  6v  +  rfyw  6w  =  0  ; 
and ,  for  <r  =  3 , 

(5.4)  fwu  6'u  +  fwy  6v  +  fww  6w  =  0. 

14  '  • 


Comment)   do1  i§  an  or4in»ry  differential,  but  feu  IB  ft 
function  of  t.  If  f  Is  Independent  of  u,  several  of  the 
terms  drop  out.   In  any  case,  If  the  determinant 

(5.5)  det(P.Par) 

Is  not  zero,  then  we  get  6u,6v,6w  In  terms  of  dc-j»dc2  (, 
In  equations  of  the  form 

6u  =  S11dc1  +  S12dc2 
6v  =  S21dc1  +  S22dc2 
6w  =  S31dc1  +  S32dc2 

where  Sj  .  Is  a  function  of  t.   (If  the  determinant  vanishes, 
the  strong  Legendre  condition  Is  not  satisfied  and  the  routine 
may  break  down.)  When  these  are  put  Into  equation  (3.1*0  for 
tfx(T) ,  we  get 


rT 


(5.6)     6x(T)  = 


^•(is"+|fs^+f  S3l)4ttol 


0 

T 


P   (d£  S   +  dv  S   +  ^  S   )dt  d02 


r 


rT 

P1-!?  S04  dt  do,  , 


c"   ""  ~wl 


— 1   —2 
with  a  similar  expression,  P  -^  P   for  6y(T).  These  have 

the  form 

2 


(5.7)  dx1(T)  =r  a1Jdc, 

1      J 


Let  us  calculate,  along  with  the  course,  a  fundamental  set  of 
solutions  to  the  adjoint.  We  also  solve  equations  (5.2)^5.3), 
(5.^)  and  substitute  the  result  Into  (5.6)  and  Its  mate  for 
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6y(T) ,  to  generate  the  Integrals  for  a  ^  in  ($•?)•     Slnoe 
a  •*  is  dx  (T)/dc,,  we  have  the  mathematical  mechanism  for 
a  Newton-Raphson  Iteration.  That  Is,  to  solve,  we  replace 
dx   by  xT  -  x  (T) ,  the  desired  minus  the  calculated  value, 
and  generate  corrections  for  c,  ,o2* 

/-  xT  -  x(T)  =  a11  Acr  +  a12  c&z 

(5*8)      \  71  99 

UT  "  y(T)  =  a^A^  +  a"  Ac2 

This  gives  a  new  estimate  of  c,,c2. 

The  principal  problem,  as  usual  with  a  Newton-Raphson 
Iteration  is  that  of  convergence.  We  may  often  force  this 
as  follows.  Kee  track  of  the  error 

(5.9)     E  =  [xT  -  x(T)]2  +  [yT  -y(T)]2 

If  this  does  not  diminish  on  successive  iterations,  divide 
the  corrections  obtained  from  (5.8)  by,  say,  a  factor  of 
five.  The  Newton-Raphson  Iteration  always  moves  the  solution 
In  the  right  direction  to  Improve  the  solution,  but  it  may 
overshoot  badly.  Once  the  values  get  close  to  the  solution 
they  tend  to  converge  rapidly. 
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6.  Control  Variable  on  the  Boundary,  Corners, 

In  the  above,  it  was  generally  assumed  that  the  minimum 
value  of  H  was  attained  for  an  interior  value  of  w,  neither 
the  maximum  nor  minimum  value.   If  the  minimum  value  of  H 
occurs  for  w  =  w"   .  then  equation  (5»M  must  be  replaced  by 

(6,1)     6w  =  0 

This  may  hold  only  in  a  certain  intervals,  or  it  may  occur 
for  the  entire  route;  the  variations  are  given  by  (5»2),(5i3) 
and  (6.1)  so  long  as  w  is  on  the  boundary.  So  long  as  the 
variables  'are  continuous,  there  is  no  particular  problem;  an  < 
extra  test  must  be  put  in  to  determine  whether  the  maximum  is 
on  the  boundary,  and  when  to  leave  it,  etc. 

However,  it  may  be  that  in  some  cases  the  minimizing 
value  of  u,v,w  is  discontinuous.   For  example,  if  the 
listening  devices  are  primarily  on  the  surface  initially, 
and  in  the  later  part  of  the  route  are  deep,  the  submarine 
may  well  need  to  change  from  W    to  some  lower  value  (a 

shallower  route),   A  oorner  can  also  occur  under  other  cir- 

- 


cumstances.  The  first  condition  for  a  corner  is  that  there  is 
a  point  t,   of  the  route  where  two  values  of  the  control , 
say,  u'.v'jW*   and  u",vK,w"   both  minimize  the  Hamiltonian 
H,   That  is,  at  this  point  t,   of  the  route 

(6.2)     H(X,5l,?,t)  -  H(X,U",P,t)  ^  mj,n:      (t  «  t,  ) 

•U  i 

Further,  on  the  route, 

.  .:     H(X,U',P,t)  >  H(X,U»,P,t)   for  t  <  t. 

<6-3)       _  _,  _       .  „  m  1 

H(X,U',P,t)  <  H(X,tf",P,t)   for  t±<   t 
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in  some  neighborhood  of  t, •   If  we  try  to  solve  for  the 
variables  by  the  minimum  principle,  we  will  get  at  least 
three  sets  of  solutions,  since  there  is  usually  a  saddle 
point  or  a  maximum  associated  with  two  relative  minima  of 
a  continuous  function. 

This  brings  up  the  following  difficulty.  There < is  no 
general  search  procedure  for  finding  the  minimum  of  a  func- 
tion of  several  variables.  It  seems  imperative  that  whoever 
programs  the  problem  must  have  some  feel  for  the  likelihood 
of  a  corner,  and  have  an  idea  where  to  look  for  the  other 
minimum.  A  routine  relying  on  gross  computation  will  surely 
require  excessive  computation. 

However,  let  us  assume  that  we  have  a  corner  and  we  have 
located  the  two  values  U1 ,  U"  for  the  control  which  yield 
equal  values,  in  (6.2).  Let  us  consider  how  a  change  in  c,, 
c2  will  change  the  Hamiltonlan  H.  The  change  will  be  due 
to  three  terms.  First,  at  a  given  point,  H  changes  directly 
with  C^.Cg.  Second  changing  c,,c2  will  change  U,  in  ac- 
cordance with  (5.2)  ,(5. 3), (5.^),  or  if  w  =  Wm  .  (5A)  Is 
replaced  by  (6.1).   Finally,  if  t,   is  changed,  there  is 
a  term  §jjdtx.  On  an  extremal.  g§  -  ||  +  Z  M^l  +r  JH^ 

+  r  —  u^s  tt,  when  the  original  equations,  and  the  Euler*- .-.* 
<y  du 

Lagrange  equations  are  used.   Hence  d>H(X,U*  ,P,t)  can  be 
put  In  the  form 

(6 A)  dH»  =  b[te1   +  4£dc2  +  dH'/et  dt1# 


_,   _  Cand  not  a  derivative 
the  prime  (')  indicating  that  U  =  U',^  There  is  a  similar 
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expression  for  dH"   at  dt^.   At  the  corner  the  two  valuet 
of  H  must  be  equal  after  the  change  In  c^Cg*  and  t^. 
Hence  we  get  an  equation  which  has  the  form 

(6.5)  ijd^  +  d>£dc2  +H'dt1  =  ijdOj^  +  d>5<ic2  +  fl'dtj  • 

Hence,  if  we  may  assume,  in  light  of  (6.3)  that 

(eHVat).  >  (tiLY/btU   , 

tl  *1 

we  can  solve  (6.5)  for  dt,  in  terms  of  do^do,.  This 
furnishes  the  value  to  put  into  the  equations  for  6x(T), 
6y(T).   In  equation  (5»6)  there  is  adjoined  a  term 

-Cp^fL  „  at, 

for  the  corner  and  dt,   is  expressed  in  terms  of  do,,dc2» 
When  these  terms  are  added  to  each  equation  in  (5»7)»  they 
are  unchanged  in  form. 

This  shows  then  how  the  routine  is  amended  to  handle 
corners . 

We  have  not  considered  the  way  in  which  the  likelihood 
of  detection  may  vary  with  the  heading  angle.  For  any  par- 
ticular listening  device,  it  may  be  a  function  of  heading 
angle  and  if  the  signal  strength  varies  greatly  as  the  sub- 
marine turns ,  there  may  be  corners  as  a  consequence  of  this. 
These  should  be  easy  to  check  for,  since  the  signal  is  probably 
symmetric  with  respect  to  the  centerplane  of  the  submarine. 
Fortunately,  if  a  corner  is  missed  by  a  small  amount,  the 
probability  of  detection  is  not  inoreased  much,  since  H  and  f 
must  have  the  same  value  on  both  sides  of  the  corner. 
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7.  Comments. 

The  problem  discussed  so  far  is  typical  and  indicates 
the  principal  points.  My  colleague  Professor  W.  E.  Blelck 
has  made  up  Fortran  programs  for  some  of  the  simpler  problems 
and  has  run  them,  without  any  particular  difficulties  arising. 

Problems  wherein  the  total  time  is  free  (that  is,  not 
specified) ,  and  wherein  the  final  point  is  not  given  but  only 
required  to  ""lie  in  a  given  region,  are  more  tedious  to  program 
but  are  not  essentially  more  difficult.   The  conditions  on  the 
route  are  not  affected  by  the  endcontitions  except  as  the  con- 
stants of  Integration  in  the  adjoint  system  are  changed. 

It  was  initially  hoped  that  a  very  general  routine  could 
be  made  up,  one  which  would  include  all  likely  difficulties, 
with  a  subroutine  adjoined  to  resolve  each  of  them.   It  is  not 
clear  now  whether  this  is  feasible. 

The  principal  difficulty  lies  in  the  nature  of  f  and  the 
way  that  it  is  generated  and  fitted.   In  the  first  place  f 
will  generally  Involve  terms  which  are  got  from  raw -data,  a 
collection  of  measurements.   An  examination  of  relevant  data 
from  Fleet  Numerical  Weather  Facility  suggests  that  there  may  be 
some  problem  in  making  up  functions  which  have  the  required 

continuous  derivatives  with  respect  to  the  control  variables. 
This  part  of  the  problem  needs  investigation.   Probably  most  of 
the  actual  work  of  getting  the  routine  to  run  and  to  yield 
good  routes  is  in  the  subproblera  of  collecting  the  data  and 
fitting  it,  so  that  the  necessary  accuracy  is  attained  without 
excessive  computational  time. 

If  the  data  Is  too  rough  and  Irregular,  it  may  be 
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necessary  to  go  to  steepest  asoent  methods  and  hope  that  there 
will  be  no  corners.  After  the  route  Is  determined,  we  may 
check  the  corner  condition,  but  this  Is  time  consuming.  And 
If  corners  then  need  occur,  the  steepest  ascent  routine  has 
an  element  of  hit  and  miss  In  locating  them. 

A  general  treatment  of  corners  Is  out  of  the  question  In 
programming,  but  that  is  not  really  needed  here.  The  function 
f  represents  physical  phenomena,  and  hence  it  will  have  some 
mathematical  structure.   If  it  is  likely  to  have  several  rela- 
tive minima,  the  people  who  work  In  the  corresponding  field 
will  surely  know  this,  and  they  must  coordinate  their  work  with 
the  programmer  so  that  he  will  know  about  when  and  where  to  look 
for  these,  and  can  organize  the  search  program  for  them.  This 
problem  is  not  simple. 

It  seems  necessary  also  that  the  programmers  of  these 
problems  have  some  understanding  6f  the  theory;  none  of  these 
problems  turn  out  to  be  routine. 

The  optimum  path  generally  has  a  nice  property:   If  the 
submarine  deviates  from  it  by  "small"  amounts,  the  resulting 
route  is  "almost  as  good",  since  the  route  gives  a  stationary 
value  to  the  probability  and  the  first-order  effects  of  changes 
are  zero. 

So  far  as  the  writer  knows,  there  is  no  general  treatment 
in  the  literature  of  the  numerioal  solution  of  problems  where 
the  control  lies  on  the  boundary  of  the  region  of  allowability, 
though  some  problems  of  this  type  have  been  solved  by  the  writer 
and  others.   It  was  necessary  in  discussing  solutions  to  examine 
this  problem  in  detail  and  to  examine  the  idea  of  normality  and 
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attainability.  Essentially  these  are  questions  otinoernlng  the 
conditions  under  which  a  point  can  be  attained,  and  under  which 
all  neighboring  endpoints  can  be  attained,  and  of  uniqueness. 
To  answer  these,  it  is  necessary  to  study  the  rank  of  certain 
functional  matrices ,  matrices  whose  elements  are  integrals 
(functionals)  with  arguments  obtained  from  the  route.   It  was 
planned  to  include  this  as  an  appendix  to  this  report,  but  the 
theory  is  not  yet  complete  and  seems  to  merit  a  separate  report, 
which  is  in  preparation. 

There  are  many  interesting  ramifications  of  this  problem. 
For  example,  if  we  know  the  location  of  the  listening  devices 
accurately,  and  the  water  conditions  exactly,  then  there  are 
sometimes  blind,  or  deaf,  areas,  regions  wherein  the  submarine 
has  very  low  likelihood  of  being  detected,  though  in  neighboring 
areas  the  likelihood  is  large,  f  being  virtually  discontinuous. 
The  theory  of  solution  of  these  problems  is  not  understood. 
Similar  problems  arise  when  air  defense  has  weapons  or  radar, 
say,  with  sharply  defined  limits. 

The  route  may  be  changed  as  subsequent  data  is  obtained. 
Indeed  we  may  envision  any  route  when  followed  by  the  submarine 
as  a  sequence  of  routes,  with  continual  updating  as  subsequent 
data  becomes  available.   If  the  data  changes  slowly,  one  or 
two  iterations  of  the  numerical  routine  will  generate  the  neces- 
sary changes  in  the  course. 

The  procedures  given  here  can  be  readily  extended  to  a 
problem  such  as  that  of  sending  a  submarine  to  launch  a  missile, 
if  data  on  the  payoffs  and  risks  is  available.   The  payoff 

i 
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balances  the  likelihood  of  damage  or  destruction  to  the  target 
against  the  corresponding  quantities  for  the  submarine,.  The 
principal  difficulty,  in  the  writer's  opinion,  is  the  collection 
and  fitting  of  data  to  describe  the  situation  accurately. 

For  those  interested  in  the  earlier  work  on  ship  routing, 
it  is  summarized  up  fairly  completely  in  the  paper  in  Navigation 
[l]  (the  number  In  brackets  refers  to  references  listed  at  the 
end.   The  author  and  his  colleagues  have  made  a  considerable 
comparison  of  the  different  methods;  this  is  summarized  in  [2]. 
Some  Fortran  programs  are  available  from  Prof.  W.  E.  Blelck.  [3]* 
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